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Abstract 

We show on a 4 X 4 example that many dynamics may ehminate aU 
strategies used in correlated equilibria, and this for an open set of games. 
This holds for the best-response dynamics, the Brown-von Neumann-Nash 
dynamics and any monotonic or weakly sign-preserving dynamics satisfy- 
ing some standard regularity conditions. For the replicator dynamics and 
the best-response dynamics, elimination of all strategies used in correlated 
equilibrium is shown to be robust to the addition of mixed strategies as 
new pure strategies. 
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1 Introduction 

A number of positive connections have been found between Nash equilibria and 
the outcome of evolutionary dynamics. For instance, for a wide clas of dynamics, 
if a solution converges to a point from an interior initial condition, then this 
point is a Nash equiUbrium (WeibuU, 1995). However, solutions of evolutionary 
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dynamics need not converge and may cycle away from the set of Nash equihbria 
(Zeeman, 1980; Hofbauer and Sigmund, 1998). 

Since the set of correlated equilibria of a game is often much larger than 
its set of Nash equilibria, it might be hoped that correlated equilibria better 
capture the outcome of evolutionary dynamics than Nash equilibria. This hope 
is reinforced by the recent litterature on adaptive processes converging, in a 
time-average sense, to the set of correlated equilibria (Hart, 2005). 

It was found, however, that there are games for which, for some initial condi- 
tions, the replicator dynamics eliminate all strategies belonging to the support 
of at least one correlated equilibrium (Viossat, 2007a). Thus, only strategies 
that do not take part in any equilibrium remain, rulling out convergence of any 
kind of time-average to the set of correlated equilibria. 

The purpose of this article is to show, on a 4 x 4 example, that elimination 
of all strategies used in correlated equilibrium does not only occur under the 
replicator dynamics and for very specific games, but for many dynamics and for 
an open set of games. We also study the robustness of this result when agents 
are explicitly allowed to use mixed strategies. 

The article is organized as follows. After presenting the framework and no- 
tations, we introduce the games we consider and explain the technique used to 
show that all strategies used in correlated equilibrium are eliminated (section 
[2]) . Sections [3l [4] and [5] deal in turn with monotonic or weakly sign-preserving 
dynamics, the best-response dynamics and the Brown-von Neumann-Nash dy- 
namics. Section [6] and the appendix show that elimination of all strategies used 
in correlated equilibrium still occurs when agents are explicitly allowed to play 
mixed strategies. Section [7] concludes. 

Framework and notations. We study single-population dynamics in 
two-player, finite symmetric games. The set of pure strategies is / = {1, 2, ..,N} 
and Sn denotes the simplex of mixed strategies (henceforth, "the simplex"). 
Its vertices e^, 1 < i < A^, correspond to the pure strategies of the game. We 
denote by Xi{t) the proportion of the population playing strategy i at time t 
and by x(i) — {xi{t), ...,XAr(t)) S Sn the population profile (or mean strategy). 
We study its evolution under dynamics of type x(i) = /(x(i),U), where U = 
iuij)i<ij<N is the payoff matrix of the game. We often skip the indication of 
time. For every x in Sn, the probability distribution on / x / induced by x is 
denoted by xgjx. If A is a subset of Sn, then conv{A) denotes its convex hull. 

We assume known the definition of a correlated equilibrium distribution 
(Aumann, 1974) and, with a slight abuse of vocabulary, we write throughout 
correlated equilibrium for correlated equilibrium distribution. A pure strat- 
egy i is used in correlated equilibrium if there exists a correlated equilibrium 
fi under which strategy i has positive marginal probability (since the game is 
symmetric, whether we restrict attention to symmetric correlated equilibria or 
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not is irrelevant; see footnote 2 in (Viossat, 2007a)). Finally, the pure strategy 
i is eliminated (for a given solution x(-) of a given dynamics) if Xi(t) — > as 

t +0O. 



2 A family of games with a unique correlated 
equilibrium 

The games considered in (Viossat, 2007a) were 4x4 symmetric games with 
payoff matrix 
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with e in ]0, 1[, and < a < (1 — £)/3. The 3x3 game obtained by omitting 
the fourth strategy is a Rock-Paper-Scissors game (RPS). This game has a 
unique Nash equilibrium : (1/3,1/3,1/3), which is also the unique correlated 
equilibrium. When a = 0, the fourth strategy of the full game earns the same 
payoff as n = (1/3,1/3,1/3,0), and there is a segment of symmetric Nash 
equihbria : for every x G [n, 64] = {An -I- (1 — A)e4,A S [0,1]}, (x,x) is a 
Nash equilibrium. For a > 0, 64 earns more than n, so (64,64) is a strict Nash 
equilibrium, and the unique correlated equilibrium is 64 64. However, for a 
small enough, the best-response cycle 61 ^ 62 ^ 63 ei remains and the 
corresponding set : 

F = {x £ 54 : 2:4 = and XiX2X^ — 0}. (2) 
is asymptotically stable under the replicator dynamics 

X^{t) = X^{t) [(Ux(t)), - X(t) ■ Ux(i)] . 

It follows that there exist games for which, for an open set of initial conditions, 
the replicator dynamics eliminate all strategies used in correlated equilibrium 
(Viossat, 2007a). 

This article shows that elimination of all strategies used in correlated equi- 
librium does not only occur for non-generic games and the replicator dynamics, 
but for an open set of games and many other dynamics. This is done by showing 
that, for many dynamics, there are values of a and e such that, for every game 
in a neighborhood of H]): 

(i) the unique correlated equilibrium is 64 ® 64; 

(ii) for an open set of initial conditions, strategy 4 is eliminated. 
Point (i) is the object of the following proposition: 
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Proposition 2.1 For every e in ]0, 1[ and every a in ]0, (1 — £)/3[, every game 
in the neighborhood of ([Ip has a unique correlated equilibrium: 64 64 . 

Proof. Since the set of games with a unique correlated equihbrium is open 
(Viossat, 2007b) and game ((!]) has a unique correlated equilibrium, it follows 
that every game in a neighborhood of ([T]) has a unique correlated equilibrium. 
Since 64 ® 64 is clearly a correlated equilibrium of every game sufficiently close 
to (P), the result follows. ■ 

To prove (ii), a first method is to show that in ([1]), and every nearby game, 
the cyclic attractor of the underlying RPS game is still asymptotically sta- 
ble. This is the method we use for monotonic dynamics and for weakly sign- 
preserving dynamics. When in the underlying RPS game the attractor is not 
precisely known, but the Nash equilibrium is repelling, another method may be 
used. It consists in showing that there is a tube surrounding the segment [n, 64] 
which repels solutions and such that outside of this tube, X4 decreases along all 
trajectories. We use this method for the Brown-von-Neumann-Nash dynamics. 
For the best-response dynamics, both methods work. 



3 Monotonic or weakly sign-preserving dynam- 
ics 

We first need some definitions. Consider a dynamics of the form 

Xi = Xigi{x.) (3) 

where the functions gi have the property that J2i<£i ^i9ii^) — ^ ^'^^ ^ 
in 5*4, so that the simplex 84^ and its boundary faces are invariant. Such a 
dynamics is monotonic if the growth rates of the different strategies are ranked 
according to their payoffsQ 

g,(x) > 5j(x) <^ (Ux)j > (Ux)j Wi G L^j G /. 

It is weakly sign-preserving (WSP) (Ritzberger and WeibuU, 1995) if whenever 
a strategy earns below average, its growth rate is negative: 

[(Ux)i < X • Ux] gi{x) < 0. 

Dynamic^ of type Q implicitly depend on the payoff matrix U. Thus, a more 
correct writing of ([3]) would be: ii = Xigi{x,\J). Such a dynamics depends 

^This property goes under various names in the literature: relative monotonicity in (Nach- 
bar, 1990), order-compatibility of pve-djnamics in (Friedman, 1991), monotonicity in (Samuel- 
son and Zhang, 1992), which we follow, and payoff monotonicity in (Hofbauer and WeibuU, 
1996). 

■^Instead of dynamics of type l[3]l, Ritzberger and WeibuU (1995) consider dynamics of the 
more general type Xi = hi{x), that need not leave the faces of the simplex positively invariant. 
Thus, we only consider a subclass of their WSP dynamics. 
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continuously on the payoff matrix if, for every i in /, gi depends continuously 
on U. A prime example of a dynamics of type ([3]) which is monotonic, WSP, 
and depends continuously on the payoff matrix is the replicator dynamics. 
Finally, a closed subset C of 84^ is asymptotically stable if it is both: 

(a) Lyapunov stable: for every neighborhood iVi of C, there exists a neigh- 
borhood N2 of C such that, for every initial condition x(0) in N2, x(i) £ Ni for 
all t > 0. 

(b) locally attracting: there exists a neighborhood of C such that, for 
every initial condition x(0) in N, miucgc ||x(i) — c|| —>t^+oo (where || • || is 
any norm on ) . 

Proposition 3.1 Fix a monotonic or WSP dynamics that depends contin- 
uously on the payoff matrix. For every a in ]0, l/3[, there exists e > such that 
for every game in the neighborhood of (QJ), the set T defined by (0) is asymptot- 
ically stable. 

Proof for monotonic dynamics. Consider a monotonic dynamics Un- 
der this dynamics, for every game in the neighborhood of (H]), the set F is a 
heteroclinic cycle. That is, a set consisting of saddle rest points and the saddle 
orbits connecting these rest points. Thus we may use the asymptotic stability's 
criteria for heteroclinic cycles developed by Hofbauer (1994) (a more accessible 
reference for this result is theorem 17.5.1 in (Hofbauer and Sigmund, 1998)). 
Specifically, associate with the heteroclinic cycle F its so-called characteristic 
matrix. That is, the 3x4 matrix whose entry in row i and column j is gj{ei) 
(for i ^ j, this is the eigenvalue in the direction of Bj of the linearization of the 
vector field at e^): 
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(5i(6i) = because e.^ is a rest point of ([3])). 

Call C this matrix. If p is a real vector, let p < (resp. p > 0) mean that 
all coordinates of p are negative (resp. positive). Hofbauer (1994) shows that 
if the following conditions are satisfied, then F is asymptotically stable: 

There exists a vector p in M** such that p > and Cp < 0. (4) 

F is asymptotically stable within the boundary of 5*4. d (5) 

Therefore, to prove proposition 13. li it is enough to show that for every a in 
]0, l/3[, there exists e > such that, for every game in the neighborhood of 

•^That is, for each proper face (subsimplex) F of 54, if F n F is nonempty, then it is 
asymptotically stable for the dynamics restricted to F. 
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conditions ^ and (O are satisfied. We begin with a lemma. In the remainder 
of this section, i e {1, 2, 3} and i — 1 and i + 1 are counted modulo 3. 

Lemma 3.2 For every < a < 1/3, there exists e > such that in game {IJ), 
for every i in {1, 2, 3}, 

g4(ei) < and < gi+i{ei) < -5,;_i(ei). (6) 

Proof. For e > 0, at the vertex e^, the payoff of strategy 4 (resp. i + 1) 
is strictly smaller (greater) than the payoff of strategy i. Since the growth 
rate of strategy i at is 0, this implies by monotonicity 54 (e^) < (resp. 
(7i+i(ei) > 0). It remains to show that f/i+i(ei) < — gi_i(ei). For e = 0, we 
have: (Uei)i = (Uej)j+i > (Uei)i_i so that = gt+i{ei) > gj_i(ej). There- 
fore gi^i(ei) < —gi-i{ei) and since the dynamics depends continuously on the 
payoff matrix, this still holds for small positive e. ■ 

We now prove proposition [SH] Fix a and e as in lemma [321 Note that since the 
dynamics we consider depends continuously on the payoff matrix, there exists 
a neighborhood of the game ([T|) in which the strict inequalities ^ still hold. 
Thus, to prove proposition 13. li it suffices to show that ([6]) implies Q and ([5]). 

dSD ^ dH) : It follows from © that if Pi = _P2 = P3 = 1 and ^4 > 0, then 
Cp < 0. Therefore, condition ^ is satisfied. 

(j6|) => {[5]) : We use again characteristic matrices. Let C denote the 3x3 
matrix obtained from C by omitting the fourth column. This corresponds to the 
characteristic matrix of F, when viewed as a hcteroclinic cycle of the underlying 
3x3 RPS game. In this RPS game, the set F is trivially asymptotically stable 
on the relative boundary of S3 (F is the relative boundary!). Furthermore, for 
p = (1/3, 1/3, 1/3) > 0, the last inequality in ([6]) implies that Cp < 0. There- 
fore, it follows from theorem 1 of Hofbauer (1994) that, in the 4x4 initial game, 
F is asymptotically stable on the face spanned by ei , 62 , 63 . Asymptotic stabil- 
ity on the face spanned by 61,61^1,64 follows easily from the following facts : 
on this face, Gi+i is a sink, a saddle, every solution starting in ]ei,ei+i] con- 
verges to e^-i-i, and x(i) depends smoothly on x(0). This concludes the proof. 



Proof of proposition 13.11 for WSP dynamics. The proof is exactly the 
same, except for the proof of lemma 13. 2i which is as follows: Fix a WSP dy- 
namics ([3]). For concreteness, set i = 2. At 62, strategy 4 earns less than average. 
Therefore (?4(e2) < 0. Now consider the case e — 0: at every point x in the 
relative interior of the edge [61,62], strategy 3 earns strictly less than average 
hence its growth rate is negative. By continuity at 62 this implies (73(62) < 0. 
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Since at 62, strategy 1 earns strictly less than average, it follows that 51(62) < 0, 
hence (73(62) < —(71(62). Since the dynamics depends continuously on the payoff 
matrix, this still holds for small positive e. 

To establish ([B]), it suffices to show that 53(62) is positive for every sufficiently 
small positive e. Let e > 0. If A > is sufficiently small then, for all /i > small 
enough, the unique strategy which earns weakly above average at x = [X^,, 1 — 
/X — A/x, /x, 0) is strategy 3, hence 5i(x) < for i ^ 3. Since X)i<i<4 ^i.9i(^) = Oj 
it follows that xigi(x) + a;353(x) > 0, hence Xfigi{:x.) + ^g^{'x) > 0, hence 
53(x) > — A5i(x). Letting ^ go to zero, we obtain 53(62) > — A5i(62) > 
(51(62) < was proved in the previous paragraph). ■ 



4 Best-response dynamics 
4.1 Main result 

The best- response dynamics (Gilboa and Matsui, 1991; Matsui, 1992) is given 
by the differential inclusion: 

x(i) e Bi?(x(t)) -x(t), (7) 

where BR{x) is the set of best responses to x: 

BR(x) = {y e Sn -.y ■ Ux = max z • Ux}. 

A solution x(-) of the best-response dynamics is an absolutely continuous func- 
tion satisfying ([7]) for almost every t. For the games and the initial conditions 
that we will consider, there is a unique solution starting from each initial con- 
ditionEI 

Consider a 4 x 4 symmetric game with payoff matrix U. Let 



V(x.) :— max 

l<i<3 



(Ux),; - ^ UiiXi 



and T/F(x) := max(a;4, |F(x)|). (8) 



l<i<4 

For every game sufficiently close to ([T]), the set 

ST := {x e S'4 : VK(x) = 0} (9) 

is a triangle, which, following Gaunersdorfer and Hofbauer (1995), we call the 
Shapley triangle. 

Proposition 4.1 For every game sufficiently close to {I]), if strategy 4 is not 
a best response to x(0), then for all t > 0, x{t) is uniquely defined, and x{t) 
converges to the Shapley triangle (0i as i — > -l-oo. 



*We focus on forward time and never study whether a solution is uniquely defined in 
backward time. 
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Proof. We begin with a lemma, which is the continuous time version of the 
improvement principle of ( Monderer and Sela (1997) 

Lemma 4.2 (Improvement principle) Let ti < t2, let b be a best response 
to x{ti) and let b' G S4. Assume that x ~ b ~ x (hence the solution points 
towards b ) for all t in]ti,t2 [■ If b' is a best response to x(t2) then b' ■ Ub > b- Ub, 
with strict inequality if b' is not a best response to xiti). 

Proof of lemma [4. 21 Between ti and t2, the solution points towards b. There- 
fore there exists A in ]0, 1[ such that 

x(t2) = Ax(ti) + (1 - A)b. (10) 

If b' is a best response to x(i2) then (b' — b) • Ux(t2) > so that, substituting 
the right-hand-side of (jlOp for x(t2), we get: 

(1 - A)(b' - b) • Ub > A(b - b') • Ux(ti). (11) 

Since b is a best response to x(ti), the right-hand-side of pT|) is nonnegative, 
and positive if b' is not a best response to x(ti). The result follows. ■ 

Proof of proposition 14.1] for game ([1]). Fix a solution x( ) of ([7]) such 
that strategy 4 is not a best response to x(0). Note that for any x in ^4, 
BR{x) ^ conv ({ei, e2, es}), because strictly dominates (1/3,1/3,1/3,0). 
Thus, either there is a unique best response to x(0) or, counting i modulo 3, 
i?i?(x(0)) = conv {{ei,ei^i}) for some i in {1,2,3}. Assume for concreteness 
that strategy 1 is the unique best response to x(0). The solution then initially 
points towards ei, until some other pure strategy becomes a best response. Due 
to the improvement principle (lemma l4?2|) . this strategy can only be strategy 2. 
Thus, the solution must then point towards the edge [ei,e2]. Since strategy 2 
strictly dominates strategy 1 in the game restricted to {1,2} x {1,2}, strategy 
2 immediately becomes the unique best response. Iterating this argument, we 
see that the solution will point towards e2, till 3 becomes a best response, then 
towards e3, till 1 becomes a best response again, and so on. 

To show that this behaviour continues for ever, it suffices to show that the 
times at which the direction of the trajectory changes do not accumulate. This 
is the object of the following claim, which will be proved in the end: 

Claim 4.3 The time length between two successive times when the direction of 
x(t) changes is bounded away from zero. 

Now recall ([S]), and note that for game ([1]) the terms uu are zero, so that 
y(x) = maxi<j<3(Ux),. Let vit) := V"(x(t)), w{t) := W^(x(t)). When x(t) 
points towards e^ (with i in {1,2,3}), we have ±4 — — X4 and 

V (Ui), = (U(e, - x)), = -V. (12) 



8 



Therefore w — ~w. Since for almost all time t, x{t) points towards ei, 62 or 
63, it follows that w{t) decreases exponentially to 0. Therefore x(t) converges 
to the Shapley triangle. 

To complete the proof, we still need to prove claim [ 



Proof of claim 14.31 In what follows i G {1, 2, 3} and i + 1 is counted modulo 
3. Fix an initial condition and let 

git) max [(Ux(t)),-(Ux(t)),] 

l<t,j<3 

denote the maximum difference between the payoffs of strategies in {1, 2, 3}. Let 
denote the fc*'' time at which strategy i becomes a best response and choose i 
such that if < tf_^^i. Simple computations, detailed in (Viossat, 2006, p. 11-12), 
show that: 

Since e < 1, it follows that for small 5 (if), we have git'^+^) > g{t^,)\ therefore 
g{t^) is bounded away from zero. Now, since (Ux(t))i — (Ux(t))i_|_i decreases 
from g{t'l) to between if and t^j^n and since the speed at which this quantity 
varies is bounded, it follows that tf^j^ — if is bounded away from zero too. That 
is, the time length between two successive times at which the direction of x(i) 
changes is bounded away from zero. ■ 

Proof of proposition 14.11 for games close to ([T]). Counting i modulo 3, 
let at = uh - Ui-i^i and (3i = - Um, i = 1,2,3. Let i £ {1,2,3}. For 

every game sufficiently close to ([T]), and Pi are positive, aia2az > Pif32f33, 
U4i < Uii, and strategy 4 strictly dominates (1/3, 1/3, 1/3, 0). Furthermore, for 
every game satisfying these conditions, the proof of proposition 14 . 1 1 for game ([T]) 
goes through. The only differences are that equation becomes 



V = (Ux)i - ^ UjjXj = (U(ei - x))i - - ^ Ujf. 

i<i<4 \ i<i<4 

and equation (jl3p becomes 

See (Viossat, 2006) for details. This completes the proof. ■ 

Note that for every 77 > 0, we may set the parameters of ([T]) so that the 
set {x G 5*4 : 64 G i?i?(x)} has Lebesgue measure less than rj. In this sense, 
the basin of attraction of the Shapley triangle can be made arbitrarily large. 
Similarly, for the replicator dynamics, the basin of attraction of the heteroclinic 
cycle ([2|) can be made arbitrarily large (Viossat, 2007a). For additional results 
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on the best-response dynamics and of the replicator dynamics in 4 x 4 games 
based on a RPS game, see (Viossat, 2006). 

5 Brown-von Neumann-Nash dynamics 

The Brown-von Neumann-Nash dynamics (henceforth BNN) is given by: 

Xi = fci(x) - a^i ^ %(x) (14) 

where 

fc,(x) max(0, (Ux), - x ■ Ux) (15) 

is the excess payoff of strategy i over the average payoff. We refer to (Hofbauer, 2000[ 
Berger and Hofbauer, 2006) and references therein for a motivation of and re- 
sults on BNN. 

Let Go denote the game ((IJ with a — 0. Recall that Uq denotes its payoff 
matrix and n = (i, i, i, O) the mixed strategy corresponding to the Nash equi- 
librium of the underlying RPS game. It may be shown that the set of symmetric 
Nash equilibria of Go is the segment Eq — [n, 64] This section is devoted to a 
proof of the following proposition: 

Proposition 5.1 IJ C is a closed subset of S4 disjoint from Eq, then there 
exists a neighborhood of Gq such that, for every game in this neighborhood and 
every initial condition in C, 2:4 (t) ^0 as t —>■ +00. 

Any neighborhood of Gq contains a neighborhood of a game of kind ([T|) , hence 
an open set of games for which 64 64 is the unique correlated equilibrium. 
Together with proposition l5.ll this implies that there exists an open set of games 
for which, under BNN, the unique strategy played in correlated equilibrium is 
eliminated from an open set of initial conditions. 

The essence of the proof of proposition 15. II is to show that, for games close 
to Go, there is a "tube" surrounding Eq such that: (i) the tube repels solutions 
coming from outside; (ii) outside of the tube, strategy 4 earns less than average, 
hence X4 decreases. We first show that in Gq the segment Eq is locally repelling. 

The function 

Fo(x) ^ E = ^ E [™^^ (0, (Uox), - X • Uox)]' 
ie/ iei 

is continuous, nonnegative and equals exactly on the symmetric Nash equilib- 
ria, i.e. on Eq, so that Vo(x) may be seen as a distance from x to i?o. Fix an 
initial condition and let vo{t) :— Vo(x(t)). 

^The game Go has other, asymmetric cquihbria, but they will play no role. 
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Lemma 5.2 There exists an open neighborhood N^q of Eq such that, under 
BNN in the game Gq, vo{t) > whenever x{t) G Neq\EQ. 

Proof. It is easily checked that: 

n • Uqx = 64 ■ Uqx Vx e (16) 

(that is, n and 64 always earn the same payolF) and 

(x-x') •Uoe4 = (x-x') -Uon^O Vx e S'4 , Vx' e 5*4 (17) 

(that is, against 64 [resp. n], all strategies earn the same payoff). Furthermore, 
as follows from lemma 4.1 in (Viossat, 2007a), for every p in Eq and every 
X ^ Eo, 

(x-p).Uox=(x-p).Uo(x-p) = l^ ^ (x.-^^) >0. (18) 

^ 1<»<3 ^ ^ 



Hofbauer (2000) shows that the function vq satisfies 

vo = P [(q - x) • Uo(q - x) - (q - x) ■ Uox] (19) 

with X = x(t), k — ki and qi — ki/k. It follows from equation (jl7p that if 
p e £'0, then against p all strategies earn the same payoff. Therefore the second 
term (q — x) • Uox goes to as x approaches Eq. Thus, to prove lemma [^21 it 
suffices to show that as x approaches Eq, the first term (q — x) ■ Uo(q — x) is 
positive and bounded away from 0. But for y: ^ Eq, 

min (Uox)i < n • Uqx = (Uox)4 < x • UqX (20) 

l<z<3 

(the first inequality holds because n is a convex combination of ei, 62 and 03, 
the equality follows from and the strict inequality from (IT51) applied to 
p = 64). It follows from (Uox)4 < x • Uox that ^4 = hence 94 = 0; similarly, 
it follows from mini<i<3(Uox)i < x ■ Uqx that = for some i in {1,2,3}. 
Together with applied to x = q, this implies that for every p in £'0, 

(q-p)-Uo(q-p)^l^ E 

This completes the proof. ■ 

Proof of proposition 15.11 Consider first the BNN dynamics in the game 
Go . Recall lemma 15.21 and let 

0<6 < min Vb(x) (21) 

xeS4\JVe, 
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(the latter is positive because Vq is positive on S4\Eo, hence on S4\Neq, and 
because S4\Neq is compact). Note that if Vb(x) < 5 then x G N^q. Therefore it 
follows from lemma 15.21 and 6 > that 

vo{t) = 6 => vo{t) > 0- (22) 

Let 

Cs := {x G ^4 : K)(x) > 5}. 

Since (5 > 0, the sets Cs and i?o are disjoint. Therefore, by applied to 
P = e4, 

x e Q ^ (Uox)4 - X • UqX < (23) 

so that X4 decreases strictly as long as x G Cs and X4 > 0. Since, by (P^ . 
the set Cs is forward invariant, it follows that for any initial condition in Cs, 
strategy 4 is eliminated. 

Now let VVb(x) = {dVo/dxi)-^^-^^ (x) denote the gradient of Vq at x. It is 
easy to see that Vq is C^. Therefore it follows from (|22p. vo{t) — VVb(x(t)) •x(t) 
and compactness of {x G 6*4 : Vb(x) = 6} that 

3-f>0,[vn{t)=S^vo{t)>-f>0]. (24) 

Similarly, since Cs is compact, it follows from that there exists 7' > such 
that 

X G C5 (Uox)4 - X • Uox < -7' < 0. (25) 

Since x is Lipschitz in the payoff matrix, it follows from ([M)) that for U close 
enough to Uq, we still have vo(t) = S =^ vq > under the perturbed dynamics. 
Similarly, due to (|25p . we still have x G C5 (Ux)4 — x • Ux < 0. Therefore 
the above reasoning applies and for every initial condition in Cs, strategy 4 is 
eliminated. 

Note that 6 can be chosen arbitrarily small (see (HI])). Therefore, to complete 
the proof of proposition (STTJ it suffices to show that if C is a compact set disjoint 
from Eo then, for S sufficiently smaU, C C C5. But since Vq is positive on S4\Eo, 
and since C is compact and disjoint from Eq, it follows that there exists S' > 
such that, for all x in C, Vo(x) > 6'; hence, for all 6 < 6', C C Cs- This 
completes the proof. ■ 



Hofbauer (2000, section 6) considers the following generalization of the BNN 



dynamics: 

n 

x^ = f{h)-x,Y,fikj) (26) 
i=i 

where / : M+ is a continuous function with /(O) = and f{u) > for 

u > 0, and where ki is defined as in ([15]). The results of this section generalize 
to any such dynamics: 
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Proposition 5.3 Consider a dynamics of type \2b]) . If C is a closed subset 
of 5*4 disjoint from Eq, then there exists a neighborhood of Gq such that, for 
every game in this neighborhood and every initial condition in C, X4{t) ^ as 

t +CXD. 

Proof. Replace Vb(x) by Wo{x.) :— J2i -^i^ii^))' where F is an anti-derivative 
of /, and replace ki by f{ki). Let / = J2if{ki), h = f(ki)/L and f = 
(•^Okxat' ^^^^^^y' = Wo(x(t)). As shown by |Hofbauer (2000) 



wo = f' 



(f-x)-Uo(f-x)-(f-x)-Uox 



which is the analogue of (fT9|) . Then apply exactly the same proof as for BNN. 



6 Robustness to the addition of mixed strategies 
as new pure strategies 

We showed that for many dynamics, there exists an open set of symmetric 4x4 
games for which, from an open set of initial conditions, the unique strategy used 
in correlated equilibrium is eliminated. Since we might not want to rule out the 
possibility that individuals use mixed strategies, and that mixed strategies be 
heritable, it is important to check whether our results change if we explicitly 
introduce mixed strategies as new pure strategies of the game. The paradigm is 
the following (Hofbauer and Sigmund, 1998, section 7.2): there is an underlying 
normal-form game, called the base game, and a finite number of types of agents. 
Each type plays a pure or mixed strategy of the base game. We assume that 
each pure strategy of the base game is played (as a pure strategy) by at least one 
type of agent, but otherwise we make no assumptions on the agents' types. The 
question is whether we can nonetheless be sure that, for an open set of initial 
conditions, all strategies used in correlated equilibrium are eliminated. This 
section shows that the answer is positive, at least for the best-response dynamics 
and the replicator dynamics. We first need some notations and vocabulary. 

Let G be a finite game with strategy set / = {1, N} and payoff matrix U. 
A finite game G" is built on G by adding mixed strategies as new pure strategies 
if: 

First, letting /' = {1, A^, + 1, A^'} be the set of pure strategies of G' 
and U' its payoff matrix, we may associate to each pure strategy i in /' a mixed 
strategy in Sn in such a way that: 

V^ e /', Vj e /', U'e^ = p* • Up^' (27) 

where ej- is the unit vector in Sn' corresponding to the pure strategy i. 
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Second, if 1 < z < TV, the pure strategy i in the game G' corresponds to the 
pure strategy i in the base game G: 

1 < i < N ^ p' = Bi. (28) 

If /u' = l))i<k,i<N' is a probabihty distribution over /' x then it induces 
the probabihty distribution fj, on I x I given by: 

Ki,j)= E y{i,j)eixi. 

l<k,l<N' 

It follows from a version of the revelation principle (see Myerson, 1994) that, if 
G' is built on G by adding mixed strategics as new pure strategics, then for any 
correlated equilibrium /x' of G' , the induced probability distribution on / x / 
is a correlated equilibrium of G. Thus, if G is a 4 x 4 symmetric game with 
64 (8) 64 as unique correlated equilibrium, then /i' is a correlated equilibrium of 
G' if and only if, for every k, I in /' such that n'{k, I) is positive, p*^ = p' = 64. 
Thus, the unique strategy of G used in correlated equilibria of G" is strategy 4. 
We show below that: 

Proposition 6.1 For the replicator dynamics and for the best-response dynam- 
ics, there exists an open set of Ax 4 symmetric games such that, for any game 
G in this set: 

(i) 64 (8) 64 is the unique correlated equilibrium of G 

(a) For any game G' built on G by adding mixed strategies as new pure 
strategies and for an open set of initial conditions, every pure strategy k in I' 
such that P4 > is eliminated. 

(the open set of initial conditions in property (ii) is a subset of Sn' , the simplex 
of mixed strategies of G' , and may depend on G') 

6.1 Proof for the best-response dynamics 

Let G be a finite game and let G' be a finite game built on G by adding mixed 
strategies of G as new pure strategies. Associate to each mixed strategy x' in 
^jv' the induced mixed strategy x in Sn defined by: 

N' 

x:=E4p'- (29) 

k=l 

Let x'(-) be a solution of the best-response dynamics in G' and x(-) the induced 
mapping from R+ to 5jv- 

Proposition 6.2 x{-) is a solution of the best-response dynamics in G. 
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Proof. For almost all t > 0, there exists a vector b' S BR{x'{t)) such that 
X (t) = b' - x.'{t). Let b T,kei' ^'kP'' ^ ^n- It follows from ^ that: 

Af' N' 

±{t) = 5] 4p^- = ^(6^ - x'fc)p'- = b - x(t). (30) 

fe=i fc=i 

Furthermore, since b' is a best response to x'(t), it follows from (|27|) and (|28p 
that b is a best response to x(t). Together with ([30|) . this implies that, for 
almost all x G BR{x.) — x. The result follows. ■ 
Since 

a;i(t) ^ ^ (Vfc G N', [p'^ > ^ 4(t) ^ O] ) 
proposition [nH] follows from propositions 14.11 and 16.21 

6.2 Proof for the replicator dynamics 

Recall that Go denotes game ([T]) with a ~ 0. Since, as already mentioned, 
every neighborhood of Gq contains an open set of games with 64 (g) 64 as unique 
correlated equilibrium, it suffices to show that every game close enough to Go 
satisfies property (ii) of proposition 16. II This is done in the appendix. 

The intuition is the following: first note that, for a game G close to Go, the 
set r defined in ([2]) is an attractor, close to which strategy 4 earns less than 
average. Now consider a game G' built on G by adding mixed strategies as new 
pure strategies and let F' denote the subset of Sn' corresponding to F: 

F' = {x G Sn' : xi + X2 + X3 — 1 and X1X2X3 = 0} 

For an initial condition close to F': (a) as long as the share of strategies fc > 4 
remains low, the solution remains close to F'; (b) as long as the solution is close 
to F', strategy 4 earns less than average and its share decreases; (c) as long as 
the share of strategy 4 does not increase, the share of strategies k > 5 remains 
low; moreover, if the share of strategy X4 decreases, so does, on average, the 
share of each added mixed strategy in which strategy 4 is played with positive 
probability. 

Putting (a), (b) and (c) together gives the result. 

7 Discussion 

We showed that elimination of all strategies used in correlated equilibrium is a 
robust phenomenon, in that it occurs for many dynamics, an open set of games 
and an open set of initial conditions. Furthermore, at least for some of the lead- 
ing dynamics, the results are robust to the addition of mixed strategies as new 
pure strategies. Under the replicator dynamics, the best-response dynamics or 
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the Brown-von Neumann-Nash dynamics, the basin of attraction of the Nash 
equilibrium of ([1]) can be made arbitrarily small. In particular, the minimal dis- 
tance from the cyclic attractor to the basin of attraction of the Nash equilibrium 
can be made much larger than the minimal distance from the Nash equilibrium 
to the basin of attraction of the cyclic attractor. The latter would thus be 
stochastically stable in a model a la Kandori, Mailath and Rob (1993)@ These 
results show a sharp difference between evolutionary dynamics and "adaptive 
heuristics" such as no-regret dynamics (Hart and Mas-CoUel, 2003; Hart, 2005) 
or hypothesis testing (Young, 2004, chapter 8). 

Some limitations of our results should however be stressed. First, our results 
have been shown here only for single-population dynamics. They imply that 
for some games and some interior initial conditions, two-population dynamics 
eliminate all strategies used in correlated equilibriu but maybe not for an 
open set of games nor for an open set of initial conditions. 

Second, the monotonic and weakly sign-preserving dynamics of section [3] 
are non-innovative: strategies initially absent do not appear. This has the 
effect that, even when focusing on interior initial conditions, the growth of 
the share of the population playing strategy i is limited by the current value 
of this share. This is appropriate if we assume that agents have to meet an 
agent playing strategy i to become aware of the possibility of playing strategy 
i; but in general, as discussed by e.g. Swinkels (1993, p. 459), this seems more 
appropriate in biology than in economics. While our results hold also for some 
important innovative dynamics, such as the best-response dynamics and a family 
of dynamics including the Brown-von Neumann-Nash dynamics, more general 
results would be welcome. 

Third, in the games we considered, the unique correlated equilibrium is a 
strict Nash equilibrium, and is thus asymptotically stable under most reasonable 
dynamics, including all those we studied. Thus, there is still an important 
connection between equilibria and the outcome of evolutionary dynamics. 

For Nash equilibrium, these three limitations can be overcome, at least par- 
tially: there are wide classes of multi-population innovative dynamics for which 
there exists an open set of games such that, for an open set of initial condi- 
tions, all strategies belonging to the support of at least one Nash equilibrium 
are eliminated ( [Viossat, 2005[ chapter 11). Moreover, for the single-population 
replicator dynamics or the single-population best-response dynamics, there are 
games for which, for almost all initial conditions, all strategies used in Nash 

^The (unperturbed) dynamics used by Kandori, Mailath and Rob (1993) is a discrete- 
time version of the best-response dynamics, but it could easily be replaced by a discrete-time 
version of another dynamics. 

^This is because for symmetric two-player games with symmetric initial conditions, two- 
population dynamics reduce to single-population dynamics, at least for the replicator dynam- 
ics, the best-response dynamics and the Brown-von-Neumann Nash dynamics. 
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equilibrium are eliminated ( [Viossat, 2005[ chapter 12). Whether these results 
extend to correlated equilibrium is an open question. 



A Proof of proposition 16.11 for the replicator dy- 
namics 

We need to show that for every game close enough to Go, property (ii) of propo- 
sition lOl is satisfied. As in section[51 let Eq = [n, 64], with n = (1/3, 1/3, 1/3, 0). 
For X in S'4\{e4}, let 

The function V takes its maximal value 1 on i?o\{e4} and its minimal value 
on the set {x S S'4\{e4} : X1X2X3 — 0}. Fix 6 in ]0, 1[. If V^(x) < S then x ^ Eq, 
hence it follows from ([20| that, at x, strategy 4 earns strictly less than average. 
Together with a compactness argument, this implies that there exists 71 > 
such that: 

F(x) < S ^ [(Uox)4 - X • Uox < -71] . (31) 

Furthermore, it is shown in (Viossat, 2007a) that in Go, under the replicator 
dynamics, the function V^(x) decreases strictly along interior trajectories (except 
those starting in Eq). More precisely, for every interior initial condition x(0) ^ 
Eq and every t in R, the function vo{t) := V{^{t)) satisfies ■i'o(i) < 0. Together 
with the compactness of {x G 5'4\{e4}, ^(x) = 5}, this implies that there exists 
72 > such that 

VQ{t) = 5^ voit) < -72. (32) 

Fix a 4 X 4 matrix U and a solution x(-) of the replicator dynamics with payoff 
matrix U, with x(0) ^ 64. Let v{t) := V{x{t)). Thus, the difference between ?;o 
and V is that, in the definition of v, x(-) is a solution of the replicator dynamics 
for the payoff matrix U and not for Uq. Since (Ux)4 — x • Ux and x are 
Lipschitz in U, it follows from (l3T|) and ((32|) that there exists 7 > such that, 
if IIU-Uoll <7: 

V{x) <S^ [(Ux)4 - x • Ux < -7] (33) 

and 

v{t) = 5^ i{t) < -7. (34) 

Fix a game G with payoff matrix U such that | |U — Uo 1 1 < 7. Let G' be a game 
built on G by adding mixed strategies of G as new pure strategies, and let U' 
be its payoff matrix. For x' in Sn' such that x'l + x'2 + 0:3 > 0, let 



V^'(x') 
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Consider a solution x'(-) of the replicator dynamics in G' (with X^Kio ^'ii^) > 
0) and let v'{t) = V'{'x'{t)). On the face of Sn' spanned by the strategies of the 
original game: 

l<i<4 

the replicator dynamics behaves just as in the base-game. Therefore, p3p and 
([M]) imply trivially that: 



J2 x'i^l and y'(x') < 5 



l<i<4 



[(U'x')4 - x' • U'x' < -7] (35) 



and 



x'^ — 1 and v'{t) = S 



l<i<4 



v{t) < -7. 



(36) 



Now define x' S Sn' as the projection of x' on the face of Sn' spanned by the 
strategies of the original game. That is, 



X,- = 



El 



if 1 < I < 4, and x' = otherwise. 



<J<4 



Note that V(x') = ^(x'). Furthermore, a simple computation shows that 
max la;' — x'A < N' max xi. 

l<i<N' 5<k<N' 

Therefore, since (U'x')4 — x' • U'x' and the vector field x' are Lipschitz in x', 
it follows from ([55]) and ([M)) that there exist positive constants rj and 7' such 
that 



max x'l, < ri and F'(x') < 5 

5<k<N' ~ \ / - 



^ (U'x')4 - x' ■ U'x' < -7' (37) 



and 



max x'u < ri and v'(t) — 5 

5<k<N' " ~ ^ ' 



v'{t) < -7'. 



(38) 



Fix y' £ Sn' such that 

E = 1' ^'(y') < -5 and C := nnn^ y', > 0. 



There exists an open neighborhood Q of y' in ^at' such that 



Vx' e 17, 



min a;' > C/2, max x',. < C77/2, and X^'(x') < (5 
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Consider an interior solution x'(-) of the replicator dynamics in G' with initial 
condition in Q,. Recall that p*"' denotes the mixed strategy of G associated with 
the pure strategy fc of G". To prove proposition 16.11 for the replicator dynamics, 
it suffices to show that: 

Proposition A.l For all fc in {4, ...,N'} such that p'l > 0, x'i^{t) ^t^+^o 0. 
Proof. We begin with two lemmas: 

Lemma A.2 Let T > and k e {5, ...,N'}. If x'^{T) < x^(0) then x'^{T) < t]. 

Proof. By construction of G", strategy k G I' earns the same payoff as the 
mixed strategy J2i<t<4Pt^t- 

l<i<4 

Therefore, it follows from the definition of the replicator dynamics that: 
Integrating between and T and taking the exponential of both sides leads to: 



4(T)=4(o) n ("ir^y 



l<i<4 



(39) 



Noting that for 1 < i < 3, we have x'^{T) < 1, l/x^(0) < 2/C and 1 < 2/C, we 

n(i?)^n(rKI)""4- <-) 

Since furthermore a;J.(0) < Crj/2, we obtain from ([39]) and ((40|) : 



The result follows. ■ 

Lemma A. 3 For all t > 0, max^g^s ^v'} < rj and v'{t) < 5. 

Proof. Otherwise there is a first time T > such that max^g^s jv'} x'ki"^) — V 
or v'{T) = S (or both). It follows from ([37]) and the definition of the replicator 
dynamics that if < i < T then ^{t) < -7' < 0. Therefore x'^{T) < a;^(0). By 
lemma [531 this implies that max;;g|5 7v/}a;J,(r) < rj. Therefore, v'{T) = 5. 
Due to ((38l) . this implies that v'{T) < 0. Therefore, there exists a time Ti with 
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<Ti <T such that v'{Ti) > 5, hence a time T2 with < T2 < Ti < T such 
v'iT2) = 6, contradicting the minimahty of T. ■ 

We now conclude: it follows from lemma equation (|37p and the defini- 
tion of the replicator dynamics that for all < > 0, x'^{t) < exY>{—Yt)x'^{0). By 
([H]) this implies that for every k in {5, iV'}, 

> 0,4(0 <r7cxp(-ph'i). 

Therefore, if P4 > then x'i,{t) — > as t ^ +00. ■ 
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